Experience in Building a Comparative Performance Analysis Engine for a Commercial System
نویسندگان
چکیده
Performance testing is a standard practice for evolving systems to detect performance issues proactively. It samples various performance metrics that will be compared with a stable baseline to judge whether the measurement data is abnormal. This type of comparative analysis requires domain expertise, which can take experienced performance analysts days to conduct. In an effort to build an automatic solution for a leading data warehousing company to improve the comparative performance analysis efficiency, we implemented machine learning approaches proposed by existing research. But the initial result has a 86% false negative rate on average, which means the majority of performance defects would be missed. To investigate causes for this unsatisfying result, we take a step back to revisit the performance data itself and find several important data related issues that are overlooked by existing work. In this paper, we discuss in detail these issues and share our hindsights to address them. With the new learning scheme we devise, we are able to reduce the false negative rate to as low as 16% and achieve a balanced accuracy of 0.91, which enables the analysis engine to be practically adopted.
منابع مشابه
Comparative Review of the Performance Based Design of Building Structures Using Static Non-Linear Analysis, Part A: Steel Braced Frames
The objective of this review to be submitted in two independent parts, for steel frames and for RC frames, is to compare their structural performance with respect to the proposed N2-method, and so also of the consequent convenience of using pushover methodology for the seismic analysis of these structures. A preliminary investigation is presented on a pushover analysis used for the seismic perf...
متن کاملComparative Review of the Performance Based Design of Building Structures Using Static Non-Linear Analysis, Part B: R/C Frames
The objective of this review to be submitted in two independent parts, for steel frames and for RC frames, is to compare their structural performance with respect to the proposed N2-method, and so also of the consequent convenience of using pushover methodology for the seismic analysis of these structures. A preliminary investigation is presented on a pushover analysis used for the seismic perf...
متن کاملA comparative analysis of two neural network predictions for performance and emissions in a biodiesel fuelled diesel Engine
متن کامل
Comparative analysis of jatropha and karanja-based biodiesel properties, performance and exhaust emission characteristics in an unmodified diesel engine
An ever-increasing drift of energy consumption, unequal geographical distribution of natural wealth and the quest for low carbon fuel for a cleaner environment are sparking the production and use of biodiesels in many countries around the globe. In this work, jatropha and karanja biodiesels were produced from the respective crude vegetable oils through transesterification, and the different phy...
متن کاملتحلیل انرژی و اگزرژی یک موتور توربوپراپ در شرایط کاری مختلف
Exergy analysis is a method of evaluating the proportion of each process on the transmission of system availability and determining how the useful energy is loosed. In this research, a turboprop engine is simulated using GASTURB 10 commercial software which is developed based on zero-dimensional approach. The performance of turboprop engine is calculated at design point and some off-design poin...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2014